Computer Vision for Music Identification: Video Demonstration

This paper describes a demonstration video for our music identification system [5]. The goal of music identification is to reliably recognize a song from a small sample of noisy audio. For instance, a user may wish to identify the music playing on her car radio or in the background at a party. She could send a few seconds of the audio using her mobile phone to a music identification server and receive a text message with the title of the song. This problem is challenging because the recording is often corrupted by noise and because the audio sample will only match a small portion of the target song. Additionally, a practical music identification system should scale (in both accuracy and speed) to databases containing hundreds of thousands of songs. Recently, the music identification problem has attracted considerable attention [1]–[4]. However, the task remains unsolved, particularly for noisy real-world queries. At first glance, problems in the audio domain may appear to have little relevance to computer vision. The former deals with processing 1-D signals over time, while computer vision tends to focus on the interpretation of one or more 2-D images (typically captured from a 3-D scene). However, we believe that certain problems in the audio domain transform very naturally into a form that can be effectively tackled by computer vision techniques. This belief is motivated by the observation that audio researchers commonly employ 2-D time-frequency representations, such as spectrograms, when analyzing sound or speech. We cast music identification into an equivalent sub-image retrieval framework: identify the portion of a spectrogram image from the database that best matches a given query snippet. Our approach treats the spectrogram of each music clip as a 2-D image and transforms music identification into a corrupted sub-image retrieval problem. By employing pairwise boosting on a large set of Viola-Jones features [6], our system learns compact, discriminative, local descriptors that are amenable to efficient indexing. During the query phase, we retrieve the set of song snippets that locally match the noisy sample and employ geometric verification in conjunction with an EM-based “occlusion” model to identify the song that is most consistent with the observed signal. We have implemented our algorithm in a practical system that can Fig. 1